3D encoding apparatus

ABSTRACT

A 3D encoding apparatus encodes an input image and generates and outputs a three dimensional stream including a basic stream and an extended stream including a plurality of image groups. A 3D encoding apparatus includes a setter that sets a specific delay value for each image group based on the a first delay value as a delay value of the basic stream and a second delay value as a delay value of the three dimensional stream. The setter sets the specific delay value to a value not more than smaller one of the first delay value and the second delay value. The first virtual buffer and the second virtual buffer perform the buffer simulation for n+1th image group based on information about the specific delay value of the nth image group set by the setter in the basic stream and three dimensional stream.

CROSS-REFERENLE TO RELATED APPLICATIONS

This is a continuation application of International Application No.PCT/JP2011/004358, with an international filing date of Aug. 1, 2011,which claims priority of Japanese Patent Application No.: 2010-182690filed on Aug. 18, 2010, the content of which is incorporated herein byreference.

BACKGROUND

1. Technical Field

The present disclosure relates to a 3D encoding apparatus for encoding3D video signals.

2. Related Art

JP-A-2008-301532 describes a generating apparatus for generating one ora plurality of encoding stream(s) including a plurality of encodedpictures per randomly accessible access unit. Concretely, the generatingapparatus includes a first determination unit that determines a delayamount between decoding of a head picture in a decoding order anddisplay of the head picture in a display order so that the delay amountsin any two access units at which decoding is carried out sequentiallyare equal to each other, and a generating unit that encodes a pluralityof pictures included in the two access units so that the delay amount isone that is determined in the first determination unit and generatesencoding streams. The generating apparatus limits the delay amount to avalue not more than a predetermined value.

According to this configuration, encoding streams are generated so thatthe delay amounts (frame delay) in two access units (for example, clips)being target for special reproduction are equal to each other. For thisreason, when those access units are sequentially decoded, a gap does notoccur between the pictures at connecting points of the access units anda frame rate is constant. That is to say, the access units are connectedseamlessly at the time of the special reproduction. As a result,discomfort that is caused by the gap can be eliminated for users whowatch moving images on a basis of the access units.

SUMMARY

A standard for encoding input images into three-dimensional streamsincluding basic streams and extended streams is H.264/MVC encodingstandard. For example, the basic streams are obtained by encoding anyone of a left image and a right image in a 3D video, and the extendedstreams are obtained by encoding the other image in the 3D video in viewof a difference between the other image and the one image being takeninto consideration. When input images to be input are encoded intothree-dimensional streams including basic streams and extended streams,technical specification described in the BD standard (System DescriptionBlu-ray Disc Read-Only Format part 3 Audio Visual Basic Specifications)should be met. Concretely, a delay value of a basic stream to be addedto predetermined GOP (Group Of pictures) and a delay value of a threedimensional stream to be added to this GOP should be equal to eachother. Further, the delay value of the basic stream to be added topredetermined GOP, a delay value of a three dimensional stream to beadded to this GOP, and the delay value of the extended stream to beadded to this GOP should be equal to each other.

The delay values are time from arrival of first byte of data (code) at abuffer of a decoder to drawing-out of data (code) of a head picture fora decoding process in the decoder from the buffer. The delay valuesdepend on a code amount of the basic stream and the extended stream.Therefore, a quantization value for encoding of input images can becontrolled so that the delay values of the basic stream and the extendedstream are equal to each other.

However, in a method for controlling a quantization value, it isdifficult to obtain suitable image quality. In order to control thequantization value so that the code amount of the basic stream and theextended stream are completely equal to each other, a configuration of a3D encoding apparatus becomes complicated.

The present disclosure is devised in view of the above problem, andprovides a three-dimensional encoding apparatus that is capable ofmaking a delay value of a basic stream and a delay value of a threedimensional stream equal to each other in a simple configuration.Further, the present disclosure provides the three-dimensional encodingapparatus that is capable of making the delay value of the basic stream,the delay value of the three dimensional stream and a delay value of anextended stream equal to each other in a simple configuration.

Additional benefits and advantages of the disclosed embodiments will beapparent from the specification and Figures. The benefits and/oradvantages may be individually provided by the various embodiments andfeatures of the specification and drawings disclosure, and need not allbe provided in order to obtain one or more of the same.

A 3D encoding apparatus according to the present disclosure includes anencoder that encodes an input image and generates and outputs a threedimensional stream including a basic stream and an extended streamincluding a plurality of image groups, a first virtual buffer thatperforms a buffer simulation of decoding the basic stream output fromthe encoder, a second virtual buffer that performs a buffer simulationof decoding the three dimensional stream output from the encoder, afirst calculator that calculates a first delay value as a delay value ofthe basic stream for each image group based on a result of the buffersimulation performed by the first virtual buffer, a second calculatorthat calculates a second delay value as a delay value of the threedimensional stream for each image group based on a result of the buffersimulation performed by the second virtual buffer, and a setter thatsets a specific delay value for each image group based on the calculatedfirst delay value and second delay value. The setter sets the specificdelay value to a value not more than smaller one of the first delayvalue and the second delay value. The first virtual buffer and thesecond virtual buffer perform the buffer simulation for n+1th imagegroup based on information about the specific delay value of the nthimage group set by the setter in the basic stream and three dimensionalstream.

A 3D encoding apparatus having the above configuration can set aspecific delay value based on a first delay value and a second delayvalue. The 3D encoding apparatus can feed back the set specific delayvalue to a simulation at a time of generating a basic stream(hereinafter, “first BS”) and a simulation at a time of generating athree dimensional stream (hereinafter, “second BS”). Therefore, the 3Dencoding apparatus can perform the first BS and the second BS on a nextimage group based on information about the same delay values. With thesimple configuration, the first delay value and the second delay valuecan be made to be equal to each other.

Since the specific delay value is a value not more than the smaller oneof the first delay value and the second delay value, even when the delayvalues of the first BS and the second BS are replaced by the same valuesso that the buffer simulation is performed, the basic stream and thethree dimensional stream that meet the BD standard can be alwaysgenerated without breaking the first BS and the second BS.

According to the 3D encoding apparatus of the present disclosure, thethree-dimensional encoding apparatus that can make the delay value ofthe basic stream and the delay value of the three dimensional streamequal to each other can be provided by the simple configuration.Further, the three-dimensional encoding apparatus that can make thedelay value of the basic stream, the delay value of the threedimensional stream and the delay value of the extended stream equal toeach other can be provided by the simple configuration.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a 3D encodingapparatus according to a first embodiment.

FIG. 2 is a flowchart illustrating one example of an operation of the 3Dencoding apparatus according to the first embodiment.

FIG. 3 is a diagram describing a condition of a buffer underflow.

FIG. 4 is a diagram illustrating a constitution of the 3D encodingapparatus according to a second embodiment.

FIG. 5 is a diagram illustrating virtual buffer control in a firstsimple CPB 301 according to the second embodiment.

FIG. 6 is a diagram describing an operation of the virtual buffercontrol over a second simple CPB 303 according to the second embodiment.

FIG. 7 is a flowchart illustrating one example of an operation of a 3Dmoving image encoding method according to the second embodiment.

FIG. 8 is a diagram illustrating a configuration of the 3D encodingapparatus according to a third embodiment.

FIG. 9 is a flowchart illustrating one example of an operation of the 3Dencoding apparatus according to the third embodiment.

FIG. 10 is a diagram illustrating a configuration of the 3D encodingapparatus according to a fourth embodiment.

FIG. 11 is a flowchart illustrating one example of an operation of the3D encoding apparatus according to the fourth embodiment.

DETAILED DESCRIPTION

Hereinafter, a 3D encoding apparatus and a 3D encoding method accordingto embodiments of the present disclosure will be described in detailwith reference to the drawings.

(First Embodiment)

1. Outline

The 3D encoding apparatus according to a first embodiment is anapparatus for encoding 3D video signals. The 3D encoding apparatusaccording to the first embodiment is an encoding apparatus for encodinginput images and generating basic streams to be used for stereoscopicvision and extended streams different from basic videos. Further,virtual buffer control using only the basic streams and virtual buffercontrol using three dimensional streams where the basic streams and theextended streams are combined are made.

A characteristic of the 3D encoding apparatus is to obtain a specificdelay value (hereinafter, “specific delay value”) based on two delayvalues obtained by the two virtual buffer controls. The obtainedspecific delay value is used for the virtual buffer control at a time ofencoding input images to be input after the obtaining of the delayvalues and generating the basic stream and the extended stream.

Hereinafter, one example of the specific delay value is the smaller oneof the two delay values obtained by the virtual buffer controls. Thespecific delay value is not limited to this value, and a value that issmaller than the two delay values can be used.

The encoding method having a configuration using H.264/AVC will bedescribed. The encoding method to be used is not limited to H.264/AVC,and another moving image encoding method may be used. That is to say,any encoding method may be used as long as it encodes an input imageinto the basic stream and the three dimensional stream.

2. Configuration

Hereinafter, the 3D encoding apparatus according to the first embodimentwill be described with reference to the drawings.

FIG. 1 is a diagram illustrating a configuration of the 3D encodingapparatus according to the first embodiment.

As shown in FIG. 1, the 3D encoding apparatus is provided with a firstencoder 101, a first CPB 102, a first delay value calculator 103, asecond encoder 104, a second CPB 105, a second delay value calculator106, a specific delay value determiner 107 and a multiplexer 108. CPB isan abbreviated name of a Coded Picture Buffer.

The first encoder 101 encodes an input image to be input using H.264/AVCto generate a basic stream. The first encoder 101 outputs the generatedbasic stream to the multiplexer 108. Further, the first encoder 101outputs an encoding parameter generated at the time of encoding theinput image to the first CPB 102 and the second CPB 105. The firstencoder 101 outputs a code amount at the time of encoding the inputimage into the basic stream to the first CPB 102 and the second CPB 105.When the input image is encoded into the basic stream, it is preferablethat MVC (multi view coding) is employed.

The first CPB 102 is a virtual buffer for performing the virtual buffercontrol based on an encoding parameter and a code amount output from thefirst encoder 101. The encoding parameter represents information aboutencoding that includes syntax information. The first CPB 102 calculatestime information tai(n), taf(n) and tr,n(n) about basic streams based onthe following six formulas. The first CPB 102 outputs the calculatedtime information to the first delay value calculator 103. The timeinformation tai(n), taf(n) and tr,n(n) and the following six formulasare defined by the H.264/AVC encoding standard. tai(n) representsinitial arrival time, namely, information about time of arrival of firstdata composing one picture at a decoder. taf(n) represents final arrivaltime, namely, information about time of arrival of last data composingone picture at the decoder. tr,n(n) represents nominal removal time,namely, information about time when data composing one picture is takenout from a buffer of the decoder.taf(n)=tai(n)+b(n)÷BitRate[SchedSelIdx]  (Formula 1)tai(n)=Max(taf(n−1),tai,earliest(n))  (Formula 2)tai,earliest(n)=tr,n(n)−(initial_cpb_removal_delay[SchedSelIdx]initial_cpb_removal_delay_offset[SchedSelIdx])±90000  (Formula 3)tai,earliest(n)=tr,n(n)−(initial_cpbremoval_delay[SchedSelIdx]÷90000)  (Formula4)tr,n(0)=initial_cpb_removal_delay[SchedSelIdx] 90000  (Formula 5)tr,n(n)=tr,n(nb)+tc*cpb_removal_delay(n)  (Formula 6)

b(n) represents a generated code amount of an nth input image of thebasic stream. The encoding parameters in the above six formulas can beobtained in the encoding process of the first encoder 101.

The first CPB 102 performs the virtual buffer control using a specificdelay value input from the specific delay value determiner 107.Concretely, the first CPB 102 assigns the specific delay value receivedfrom the specific delay value determiner 107 toinitial_cpb_removal_delay in (Formula 4). The first CPB 102 calculatestai,earliest(n) and performs the subsequent virtual buffer control.

The first delay value calculator 103 calculates a delay value Δtg, 90(n)per GOP based on time information calculated by the first CPB 102.Δtg,90(n)=90000*(tr,n(n)−taf(n−1))  (Formula 7)

The above formula is a formula that is defined by the H.264/AVC encodingstandard.

The first delay value calculator 103 outputs the calculated delay valueto the specific delay value determiner 107.

The second encoder 104 encodes an input image to be input based on theH.264/AVC standard, generates an extended stream, and outputs thegenerated extended stream to the multiplexer 108. The second encoder 104outputs the encoding parameter generated at the time of encoding theinput image to the second CPB 105. The second encoder 104 outputs a codeamount at the time of encoding the input image into the extended streamto the second CPB 105. When the input image is encoded into the extendedstream, it is preferable that MVC (multi view coding) is employed.

The second CPB 105 is a virtual buffer for performing the virtual buffercontrol based on the encoding parameter and the code amount output fromthe first encoder 101 and the second encoder 104. The second CPB 105calculates the time information tai(n), taf(n) and tr,n(n) relating to athree dimensional stream based on the formula 1 to the formula 6. Thesecond CPB 105 assigns the encoding parameter relating to the threedimensional stream to the formulas 1 to 6. b(n) represents a sum of agenerated code amount of the nth input image in the extended stream anda generated code amount of the nth input image in the basic stream.

The second CPB 105 performs the virtual buffer control using thespecific delay value input from the specific delay value determiner 107.The second CPB 105 assigns the specific delay value received from thespecific delay value determiner 107 to initial_cpb_removal_delay in(Formula 4). The second CPB 105 calculates tai,earliest(n) and performsthe subsequent virtual buffer control.

The second delay value calculator 106 calculates a delay value Δtg,90(n) per GOP according to (Formula 7) based on the time informationcalculated by the second CPB 105. The second delay value calculator 106outputs the calculated delay value Δtg, 90(n) to the specific delayvalue determiner 107.

The specific delay value determiner 107 determines (calculates) aspecific delay value based on the delay values output from the firstdelay value calculator 103 and the second delay value calculator 106.For example, smaller one of delay values obtained by the first delayvalue calculator 103 and the second delay value calculator 106 isdetermined as the specific delay value. The specific delay valuedeterminer 107 may calculate a value smaller than the first delay valueoutput from the first delay value calculator 103 and the second delayvalue output from the second delay value calculator 106 to determine itas the specific delay value. The specific delay value determiner 107outputs the determined specific delay value to the multiplexer 108.Further, the specific delay value determiner 107 outputs the determinedspecific delay value to the first CPB 102 and the second CPB 105.

The multiplexer 108 combines the basic stream and the extended stream togenerate a three dimensional stream. At this time, the multiplexer 108writes the specific delay value output by the specific delay valuedeterminer 107 into the three dimensional stream. That is to say, themultiplexer 108 writes the specific delay value determined by thespecific delay value determiner 107 as the delay value of the basicstream. The multiplexer 108 writes the specific delay value determinedby the specific delay value determiner 107 as the delay value of thethree dimensional stream. As a result, the delay value of the basicstream and the delay value of the three dimensional stream can be madeto be equal to each other. In H.264/AVC, the delay value of the basicstream is written into “Buffering period SEI”. The delay value of thethree dimensional stream is written into “Buffering period SEI” in“scalable nesting SEI”. The above “Buffering period SEI”, “scalablenesting SEI” and “Buffering period SEI” are syntax information definedby the H.264/AVC standard.

3. Operation

One example of an operation of the 3D encoding apparatus according tothe first embodiment will be described below with reference to aflowchart of FIG. 2.

(Step S201) When an input image is input, the first encoder 101 encodesthe input image to generate a basic stream. The first encoder 101outputs the generated basic stream to the multiplexer 108. The firstencoder 101 outputs a code amount and an encoding parameter of thegenerated basic stream to the first CPB 102 and the second CPB 105.

(Step S202) The first CPB 102 performs the virtual buffer control basedon the encoding parameter and the code amount of the basic streamgenerated by the first encoder 101. The first CPB 102 outputs timeinformation generated by the process of the virtual buffer control tothe first delay value calculator 103.

(Step S203) On the other hand, when the input image is input, the secondencoder 104 encodes the input image to generate an extended stream. Thesecond encoder 104 outputs the generated extended stream to themultiplexer 108. Further, the second encoder 104 outputs a code amountand an encoding parameter of the generated extended stream to the secondCPB 105.

(Step S204) The second CPB 105 performs the virtual buffer control basedon the code amount and the encoding parameter of the basic streamgenerated by the first encoder 101, and the code amount and the encodingparameter of the extended stream generated by the second encoder 104.The second CPB 105 outputs time information generated in the process ofthe virtual buffer control to the second delay value calculator 106.

(Step S205) The first encoder 101 and the second encoder 104 determinewhether an input image to be encoded is an input image positioned at theend of GOP (hereinafter, “terminal image”). When determined asnon-terminal image, the sequence goes to step S201, and the firstencoder 101 and the second encoder 104 encode residual input images inGOP. When determined as a terminal image, the sequence goes to stepS206.

(Step S206) The first delay value calculator 103 calculates a delayvalue Δtg, 90(n) (hereinafter, suitably “a first delay value”) of thebasic stream based on time information output from the first CPB 102.The first delay value is not a specific delay value to be actually addedto the basic stream. The specific delay value is calculated by thespecific delay value determiner 107. The first delay value calculator103 outputs the calculated first delay value to the specific delay valuedeterminer 107.

(Step S207) The second delay value calculator 106 calculates a delayvalue Δtg, 90(n) (hereinafter, suitably “a second delay value”) to beapplied to the three dimensional stream based on the time informationoutput from the second CPB 105. The second delay value is not a specificdelay value. The specific delay value is calculated by the specificdelay value determiner 107. The second delay value calculator 106outputs the calculated second delay value to the specific delay valuedeterminer 107.

(Step S208) The specific delay value determiner 107 determines smallerone of the first delay value and the second delay value output from thefirst delay value calculator 103 and the second delay value calculator106 as the specific delay value, and outputs it to the first CPB 102 andthe second CPB 105.

(Step S209) Finally, the first CPB 102 and the second CPB 105 performthe virtual buffer control relating to next GOP which follows GOP ofwhich specific delay value is obtained based on the specific delay valueoutput from the specific delay value determiner 107.

The above operation flow is one example, and the respective steps can beshuffled. The specific delay value generated at S208 is not limited tothe above value, and thus, for example, a delay value smaller than thedelay values output from the first delay value calculator 103 and thesecond delay value calculator 106 may be used as a specific delay value.

4. Correspondence Between the Present Disclosure and the Embodiment

“The virtual buffer control” according to the embodiment is one exampleof “buffer simulation” in the present disclosure. “The first encoder101” and “the second encoder 104” according to the embodiment are oneexample of “an encoder” of the present disclosure. “The first CPB 102”according to the embodiment is one example of “a first virtual buffer”of the present disclosure. “The second CPB 105” according to theembodiment is one example of “a second virtual buffer” of the presentdisclosure. “The first delay value calculator 103” according to theembodiment is one example of “a first calculator” of the presentdisclosure. “The second delay value calculator 106” according to theembodiment is one example of “a second calculator” of the presentdisclosure. “The specific delay value determiner 107” according to theembodiment is one example of “a setter” of the present disclosure.

5. Conclusion

The 3D encoding apparatus according to the embodiment includes the firstencoder 101 and the second encoder 104 that encode an input image, andgenerate and output a basic stream and an extended stream, the first CPB102 that performs the buffer simulation of decoding the basic streamoutput from the first encoder 101, the second CPB 105 that performs thebuffer simulation of decoding a three dimensional stream obtained fromthe basic stream output from the first encoder 101 and the extendedstream output from the second encoder 104, the first delay valuecalculator 103 that calculates the first delay value as the delay valueof the basic stream per GOP based on a result of the buffer simulationperformed by the first CPB 102, the second delay value calculator 106that calculates the second delay value as the delay value of the threedimensional stream per GOP based on a result of the buffer simulationperformed by the second CPB 105, and the specific delay value determiner107 that sets a specific delay value per GOP based on the calculatedfirst delay value and second delay value. The specific delay valuedeterminer 107 sets a value not more than smaller one of the delay valueof the basic stream and the delay value of the three dimensional streamas the specific delay value. The first CPB 102 and the second CPB 105perform the buffer simulation for n+1th GOP based on the specific delayvalue set by the specific delay value determiner 107 for nth GOP in thebasic stream and the three dimensional stream.

In such a manner, the 3D encoding apparatus can set the specific delayvalue based on the first delay value and the second delay value. The 3Dencoding apparatus feeds back the set specific delay value to a buffersimulation at the time of generating the basic stream (hereinafter,first BS) and a buffer simulation at the time of generating the threedimensional stream (hereinafter, second BS). Therefore, the 3D encodingapparatus can perform the first BS and the second BS on next GOP (n+1thGOP) based on the same delay values. Further, with a simpleconfiguration, the first delay value and the second delay value can beset to the same value.

Since the specific delay value is a value not more than the smaller oneof the first delay value and the second delay value, even when the delayvalues of the first BS and the second BS are replaced by the same valuesso that the buffer simulation is performed, the basic stream and thethree dimensional stream that meet the BD standard can be alwaysgenerated without interrupting the first BS and the second BS.

The first CPB 102 and the second CPB 105 output, to the first delayvalue calculator 103 and the second delay value calculator 106,information about time at which an input image is encoded as a result ofthe buffer simulation.

In such a manner, the 3D encoding apparatus can calculate the delayvalue based on the information about the time of the encoding. As aresult, since the accurate delay value is calculated so that the buffersimulation can be performed, while a deterioration in videos is beingrepressed, the basic stream and the three dimensional stream that meetthe BD standard can be generated.

(Second Embodiment)

1. Outline

In the first embodiment, the first delay value calculator 103 and thesecond delay value calculator 106 calculate the delay values based onthe time information output from the first CPB 102 and the second CPB105. Even when the first delay value output from the first delay valuecalculator 103 and the second delay value output from the second delayvalue calculator 106 are replaced by the same values, the virtual buffercontrol is adjusted so that the basic stream and the three dimensionalstream that meet the BD standard are generated without breaking therestriction of the H.264/AVC standard. In the first embodiment, thevirtual buffer control is made by using the time information in such amanner.

The virtual buffer control that is performed by the first CPB 102 andthe second CPB 105 is used for ensuring non-occurrence of a bufferunderflow defined by the H.264/AVC standard.

FIG. 3 is a diagram describing a situation of the buffer underflow. Thebuffer underflow means that a buffer accumulation amount (hereinafter,“cpb value”) in the virtual buffer control obtains a negative value.When a code amount exceeds the cpb value at a time of starting theencoding of an input image in the first encoder 101 and the secondencoder 104, in other words, when a taking-out amount exceeds the cpbvalue at the time of starting to encode an input image in a virtualbuffer modeled on an input buffer of the decoder (a code amount is notless than a takable-out amount at the time of starting to encode aninput image), the buffer underflow occurs. For this reason, the firstencoder 101 and the second encoder 104 set a code amount smaller thanthe cpb value at the time of starting to encode an input image as amaximum code amount of the input image. The first encoder 101 and thesecond encoder 104 adjust the encoding parameter so that the generatedcode amount does not exceed the maximum code amount at a time of actualencoding.

When the above point is taken into consideration, the use of the cpbvalue performs the virtual buffer control easier than use of timeinformation. When a CABAC (context adaptive binary arithmetic coding)process is used in the H.264/AVC standard, a delay occurs until thegenerated code amount of each input image is determined. For thisreason, the cpb value cannot be obtained at suitable timing. Also due tothis, not the virtual buffer control based on the time by means of thedelay values but the simple virtual buffer control based on the cpbvalue is necessary. For this reason, the virtual buffer control based onthe cpb value is occasionally used as the virtual buffer control. Thecpb value in the case should be modified in order to keep therestriction of the BD standard.

A second embodiment provides the 3D encoding apparatus that is capableof making the delay value of the basic stream and the delay value ofthree dimensional stream equal to each other even when the cpb value isused for the virtual buffer control.

In the second embodiment, the delay value of the basic stream and thedelay value of the three dimensional stream are calculated based onencoding rates and the cpb values of the basic stream and threedimensional stream. The specific delay value is set based on thecalculated two delay values.

2. Configuration

A configuration of the 3D encoding apparatus according to a secondembodiment will be described below with reference to the drawings.

FIG. 4 is a diagram illustrating the configuration of the 3D encodingapparatus according to the second embodiment. The parts of theconfiguration similar to those in the first embodiment are denoted bythe same reference symbols, and detailed description thereof is omitted.

The first simple CPB 301 is a virtual buffer for performing the virtualbuffer control using the cpb value based on the encoding parameter andthe code amount output from the first encoder 101. The encodingparameter represents information about the encoding including syntaxinformation. The code amount is a code amount obtained by actuallyencoding an input image in the first encoder 101 or a code amount as apredicted value in the first encoder 101. The first simple CPB 301calculates the cpb value (hereinafter, “cpb1”) in the basic stream. Thefirst simple CPB 301 calculates cpb1(n) in nth input image according tothe following formula.cpb1(n)=cpb1(n−1)+(BR1*T)−b1(n)  (Formula 8)

BR1 represents a maximum bit rate of the basic stream, and is, forexample, 12 Mbps. T is an inverse number of a picture rate. In otherwords, T is a value representing time from (n−1st) input image includedin the basic stream to nth input image. Further, b1(n) represents a codeamount of the nth input image in the basic stream. The maximum bit rateand the picture rate of the basic stream are given from a controller 400that controls the 3D encoding apparatus entirely. The maximum bit rateof the basic stream may be a value that changes in each GOP. Theconfiguration may be such that the fixed maximum bit rate is given toinput images.

The first simple CPB 301 outputs cpb1(n) to a first delay valuecalculator 302. The first simple CPB 301 outputs cpb1(n) to the firstdelay value calculator 302 at, for example, the end of GOP.

The first simple CPB 301 uses the cpb value output from a CPB valueinverse converter 306 for the virtual buffer control over GOP (n+1thGOP) next to GOP (nth GOP) of which first delay value and second delayvalue are obtained. That is to say, in the virtual buffer control overGOP next to GOP that is currently processed, the cpb value output by theCPB value inverse converter 306 is used instead of the current cpbvalue. Even when the current cpb value is replaced, the delay value ofthe basic stream and delay value of the three dimensional stream can bemade to be equal to each other by the above operation of the firstsimple CPB 301 without interrupting the virtual buffer control.

The first delay value calculator 302 calculates the delay value of thebasic stream based on cpb1 output from the first simple CPB 301 and themaximum bit rate of the basic stream given from the controller 400 thatcontrols the 3D encoding apparatus entirely.

FIG. 5 is a diagram illustrating the virtual buffer control in the firstsimple CPB 301.

In the graph of FIG. 5, a gradient shown by a broken line represents themaximum bit rate. In this case, a relationship between the cpb value andthe delay value is as shown in FIG. 5. For example, 1500 bits is givenas the cpb value, and 12 Mbps is given as the maximum bit rate. In thiscase, the first delay value calculator 302 calculates the delay value toobtain 0.125 ms according to the following formula.(delay value)=(the cpb value)/(the maximum bit rate)  (Formula 10)

The first delay value calculator 302 outputs the calculated delay valueto the specific delay value determiner 305.

The second simple CPB 303 is a virtual buffer for performing the virtualbuffer control using the cpb value based on the encoding parameter andthe code amount output from the first encoder 101 and the second encoder104. The encoding parameter represents information relating to theencoding including syntax information. The code amount is a code amountobtained by actually encoding an input image through the first encoder101 and the second encoder 104 or a code amount as a predicted value.The second simple CPB 303 calculates the cpb value (hereinafter, cpb2)in the three dimensional stream. The second simple CPB 303 calculatescpb2(n) of nth input image according to the following formula.cpb2(n)=cpb2(n−1)+(BR2*T)−b2(n)  (Formula 11)

BR2 represents a maximum bit rate of the three dimensional stream, andis, for example, 24 Mbps. T represents an inverse number of the picturerate. In short, T is a value representing the time from the (n−1st)input image to the nth input image included in the three dimensionalstream. Further, b2(n) represents a code amount of the nth input imageof the three dimensional stream. That is to say, b2(n) is a sum of thecode amount of the nth input image in the basic stream output from thefirst encoder 101, and the code amount of the nth input image in theextended stream output from the second encoder 104. The maximum bit rateand the picture rate in three dimensional stream is given from thecontroller 400 that controls the 3D encoding apparatus entirely. Themaximum bit rate of the three dimensional stream may be a value thatchanges in each GOP. The fixed maximum bit rate may be given to theinput image.

The second simple CPB 303 outputs cpb2(n) to the second delay valuecalculator 304. The second simple CPB 303 outputs cpb2 to the seconddelay value calculator 304 at, for example, the end of GOP.

The second simple CPB 303 uses the cpb value output from the CPB valueinverse converter 306 for the virtual buffer control over next GOP. Thatis to say, in the virtual buffer control over GOP (n+1th GOP) next toGOP (nth GOP) of which first delay value and second delay value used forthe calculation of the specific delay value are obtained, the cpb valueoutput from the CPB value inverse converter 306 is used instead of thecurrent cpb value. Even when the current cpb value is replaced, thedelay value of the basic stream and the delay value of the threedimensional stream can be made to be equal to each other by the aboveoperation of the second simple CPB 303 without interrupting the virtualbuffer control.

The second delay value calculator 304 calculates the delay value of thethree dimensional stream based on cpb2 output from the second simple CPB303 and the maximum bit rate of the three dimensional stream given fromthe controller 400 that controls the 3D encoding apparatus entirely.

The second delay value calculator 304 calculates the delay value of thethree dimensional stream based on the relationship in (Formula 10). Forexample, 3200 bits is given as cpb2, and 24 Mbps is given as the maximumbit rate. In this case, the second delay value calculator 304 calculatesthe delay value to obtain 0.133 ms according to the following formula.

The second delay value calculator 304 outputs the calculated delay valueto the specific delay value determiner 305.

The specific delay value determiner 305 dete mines (calculates) thespecific delay value based on the delay values output from the firstdelay value calculator 302 and the second delay value calculator 304.For example, smaller one of the delay value obtained by the first delayvalue calculator 302 and the delay value obtained by the second delayvalue calculator 304 is determined to be output as the delay value. Thespecific delay value determiner 305 may calculate to output a valuesmaller than the obtained delay value. The specific delay valuedeterminer 305 outputs the determined (calculated) delay value to theCPB value inverse converter 306. Further, the specific delay valuedeterminer 305 outputs the determined (calculated) specific delay valueto the multiplexer 108.

The CPB value inverse converter 306 calculates the cpb value employed inthe first simple CPB 301 and the cpb value employed in the second delayvalue calculator 304 based on the specific delay value output from thespecific delay value determiner 305, the maximum bit rate of the basicstream and the maximum bit rate of the three dimensional stream given bythe controller 400 for controlling the 3D encoding apparatus entirely.The CPB value inverse converter 306 outputs the calculated cpb values tothe first simple CPB 301 and the second simple CPB 303.

For example, the CPB value inverse converter 306 obtains 0.125 ms as thespecific delay value from the specific delay value determiner 305. Themaximum bit rate of the basic stream is determined as 12 Mbps, and themaximum bit rate of the three dimensional stream is determined as 24Mbps. At this time, the CPB value inverse converter 306 calculates thecpb values to be employed in the first simple CPB 301 and the secondsimple CPB 303 according to the following formula.(the cpb value)=(delay value)*(the maximum bit rate)  (Formula 12)

That is to say, the CPB value inverse converter 306 calculates the cpbvalues to be employed in the first simple CPB 301 to obtain 1500 bits.The CPB value inverse converter 306 calculates the cpb value to beemployed in the second simple CPB 303 to obtain 3000 bits. In this case,since the specific delay value output from the specific delay valuedeterminer 305 is the delay value calculated by the first delay valuecalculator 302, the cpb value output from the CPB value inverseconverter 306 to the first simple CPB 301 is not different from the cpbvalue originally output from the first simple CPB 301. However, the cpbvalue output from the CPB value inverse converter 306 to the secondsimple CPB 303 is calculated based on the delay value calculated by thefirst delay value calculator 302.

3. Operation

3-1. Operation of the Second Simple CPB 303

Before description of an entire operation, an operation of the secondsimple CPB 303 will be described with reference to the drawings.

FIG. 6 is a diagram describing an operation of the virtual buffercontrol in the second simple CPB 303.

A cpb 601 shown in FIG. 6 represents the cpb value at the end of GOP.The second simple CPB 303 outputs the cpb value to the second delayvalue calculator 304. When the second simple CPB 303 outputs the cpbvalue to the second delay value calculator 304, a modified cpb value isinput into the second simple CPB 303 via the second delay valuecalculator 304, the specific delay value determiner 305 and the CPBvalue inverse converter 306. The modified cpb value is, for example, cpb602 shown in FIG. 6. The second simple CPB 303 uses the cpb 602 in thevirtual buffer control over GOP (n+1th GOP) next to GOP (nth GOP) ofwhich specific delay value is determined. In short, the cpb value of thesecond simple CPB 303 is modified according to the change in the delayvalue of the specific delay value determiner 305.

In such a manner, the delay values of the basic stream and the threedimensional stream can be easily set to the same value. Even when thedelay value obtained from the virtual buffer control is changed, inputimages can be encoded without interrupting the subsequent virtual buffercontrol. An influence of the change in the delay values necessary on thebasis of the BD standard can be reflected on the cpb value to be usedfor avoiding the buffer underflow. As a result, while the bufferunderflow is being repressed, and the encoding operation that meets theBD standard can be realized.

3-2. Operation of the 3D Encoding Apparatus

One example of an operation of a 3D moving image encoding methodaccording to the second embodiment will be described below withreference to a flowchart of FIG. 7.

(Step S701) When an input image is input, the first encoder 101 encodesthe input image to generate a basic stream. The first encoder 101outputs the generated basic stream to the multiplexer 108. The firstencoder 101 outputs a code amount and an encoding parameter of thegenerated basic stream to the first simple CPB 301 and the second simpleCPB 303.

(Step S702) The first simple CPB 301 performs the virtual buffer controlusing the cpb value based on the encoding parameter and the code amountof the basic stream generated by the first encoder 101.

(Step S703) On the other hand, when an input image is input, the secondencoder 104 encodes the input image to generate an extended stream. Thesecond encoder 104 outputs the generated extended stream to themultiplexer 108. The second encoder 104 outputs a code amount and anencoding parameter of the generated extended stream to the second simpleCPB 303.

(Step S704) The second simple CPB 303 performs the virtual buffercontrol using the cpb value based on the code amount and the encodingparameter of the basic stream generated by the first encoder 101 and thecode amount and the encoding parameter of the extended stream generatedby the second encoder 104.

(Step S705) The first encoder 101 and the second encoder 104 determinewhether the input image to be encoded is an input image (hereinafter, “aterminal image”) positioned at the end of GOP. When the determination ismade as not being a terminal image, the sequence goes to S701, and thefirst encoder 101 and the second encoder 104 encodes residual inputimages in GOP. When the determination is made as being a terminal image,the first simple CPB 301 outputs the cpb value at the time of the end ofGOP to the first delay value calculator 302. The second simple CPB 303outputs the cpb value at the time of the end of GOP to the second delayvalue calculator 304. The sequence, then, goes to S706.

(Step S706) The first delay value calculator 302 calculates a delayvalue (hereinafter, “the first delay value”) of the basic stream basedon the cpb value output from the first simple CPB 301. The first delayvalue is not the specific delay value to be actually added to the basicstream. The specific delay value to be actually added is calculated bythe specific delay value determiner 305. The first delay valuecalculator 302 outputs the calculated first delay value to the specificdelay value determiner 305.

(Step S707) The second delay value calculator 304 calculates a delayvalue (hereinafter, “second delay value”) to be applied to the threedimensional stream based on the cpb value output from the second simpleCPB 303. The second delay value is not the specific delay value. Thespecific delay value to be actually added is calculated by the specificdelay value determiner 305. The second delay value calculator 304outputs the calculated second delay value to the specific delay valuedeterminer 305.

(Step S708) The specific delay value determiner 305 determines a smallerone of the first delay value and the second delay value output from thefirst delay value calculator 302 and the second delay value calculator304 as the specific delay value, and outputs it to the CPB value inverseconverter 306.

(Step S709) The CPB value inverse converter 306 calculates the cpbvalues to be used by the first simple CPB 301 and the second simple CPB303 based on the specific delay value output from the specific delayvalue determiner 305. The CPB value inverse converter 306 outputs thecalculated cpb values to the first simple CPB 301 and the second simpleCPB 303.

(Step S710) Finally, the first simple CPB 301 and the second simple CPB303 perform the virtual buffer control over GOP (n+1th GOP) next to GOP(nth GOP) of which specific delay value is obtained based on the cpbvalue output from the CPB value inverse converter 306.

The above operation flow is one example, and the respective steps can beshuffled. The specific delay values generated at S708 are not limited tothe above values and for example, a delay value that is smaller than thefirst delay value and the second delay value output from the first delayvalue calculator 302 and the second delay value calculator 304 may beused as the specific delay value.

4. Conclusion

The 3D encoding apparatus according to the embodiment includes the firstencoder 101 and the second encoder 104 that encode an input image togenerate a basic stream and an extended stream, the first simple CPB 301that performs the buffer simulation of decoding the basic stream outputfrom the first encoder 101, the second simple CPB 303 that performs thebuffer simulation of decoding a three dimensional stream based on thebasic stream output from the first encoder 101 and the extended streamoutput from the second encoder 104, the first delay value calculator 302that calculates the first delay value as a delay value of the basicstream for each GOP based on a result of the buffer simulation performedby the first simple CPB 301, the second delay value calculator 304 thatcalculates the second delay value as a delay value of the threedimensional stream for each GOP based on a result of the buffersimulation performed by the second simple CPB 303, and the specificdelay value determiner 305 that sets the specific delay value for eachGOP based on the calculated first delay value and second delay value.The specific delay value determiner 305 sets a value that is not morethan smaller one of the delay value of the basic stream and the delayvalue of the three dimensional stream as the specific delay value. Thefirst simple CPB 301 and the second simple CPB 303 perform the buffersimulation of n+1th GOP based on the specific delay value of nth GOP inthe basic stream and three dimensional stream set by the specific delayvalue determiner 305. When the specific delay value output from thespecific delay value determiner 305 is input, the first simple CPB 301modifies the buffer accumulation amount of the first simple CPB 301 atthe input time based on the specific delay value, and performs thebuffer simulation of GOP (n+1th GOP) next to GOP (nth GOP) of whichspecific delay value is obtained. When the specific delay value outputfrom the specific delay value determiner 305 is input, the second simpleCPB 303 modifies a buffer accumulation amount of the second simple CPB303 at the input time based on the specific delay value, and performsthe buffer simulation of GOP (n+1th GOP) next to GOP (nth GOP) of whichspecific delay value is obtained.

In such a manner, even when the 3D encoding apparatus performs thebuffer simulation using the buffer accumulation amount, the delay valueto be added to the basic stream and the delay value to be added to thethree dimensional stream can be made to be equal to each other by thesimple configuration. As a result, even when the buffer simulation isperformed in the simple configuration, the basic stream and the threedimensional stream that meet the BD standard can be generated.

Since the specific delay value is not more than smaller one of the firstdelay value and the second delay value, even when the delay values ofthe first BS and the second BS are replaced by the same delay value sothat the buffer simulation is performed, the basic stream and the threedimensional stream that meet the BD standard can be always generatedwithout interrupting the first BS and the second BS.

When the specific delay value output from the specific delay valuedeterminer 305 is input to the first simple CPB 301, the first simpleCPB 301 modifies the buffer accumulation amount of the first simple CPB301 at the input time based on the specific delay value, and performsthe buffer simulation of GOP (n+1th GOP) next to GOP (nth GOP) of whichspecific delay value is obtained. When the specific delay value outputfrom the specific delay value determiner 305 is input, the second simpleCPB 303 modifies the buffer accumulation amount of the second simple CPB303 at the input time based on the specific delay value, and performsthe buffer simulation of GOP (n+1th GOP) next to GOP (nth GOP) of whichspecific delay value is obtained.

In such a manner, the buffer accumulation amounts in the first simpleCPB 301 and the second simple CPB 303 can be modified according to thesetting of the specific delay value. As a result, even when the firstdelay value and the second delay value are changed into the specificdelay value, the 3D encoding apparatus can generate the basic stream andthe three dimensional stream that meet the BD standard withoutinterrupting the buffer simulation using the buffer accumulation amount.

(Third Embodiment)

1. Outline

The first embodiment describes the configuration where the specificdelay value is calculated so that the delay value of the basic streamand the delay value of the three dimensional stream calculated based onthe time information are equal to each other.

In the third embodiment, the specific delay value is calculated so thatthe delay value of the basic stream, the delay value of the extendedstream and the delay value of the three dimensional stream calculatedbased on the time information are equal to each other.

2. Configuration

The configuration of the 3D encoding apparatus according to a thirdembodiment will be described below with reference to the drawings.

FIG. 8 is a diagram illustrating the configuration of the 3D encodingapparatus according to the third embodiment. Parts that are the same asthose in the first embodiment are denoted by the same reference symbols,and detailed description thereof is omitted.

The first delay value calculator 109 basically performs the similaroperation to that of the first delay value calculator 103 in the firstembodiment. However, the first delay value calculator 109 is differentfrom the first delay value calculator 103 in that the calculated delayvalue is output to the specific delay value determiner 113.

The second delay value calculator 110 basically performs the similaroperation to that of the second delay value calculator 106 in the firstembodiment. The second delay value calculator 110 is different from thesecond delay value calculator 106 in that the calculated delay value isoutput to the specific delay value determiner 113.

A third CPB 111 is a virtual buffer for performing the virtual buffercontrol based on the encoding parameter and the code amount output fromthe second encoder 104. The third CPB 111 calculates time informationtai(n), taf(n), tr,n(n) relating to the extended stream based on theformulas 1 to 6. The time information tai(n), taf(n), tr,n(n), and theformulas 1 to 6 are defined by the H.264/AVC standard as described inthe first embodiment. The third CPB 111 assigns the encoding parameterrelating to the extended stream output from the second encoder 104 tothe formulas 1 to 6. Here, b(n) is the generated code amount of the nthinput image in extended stream.

The third CPB 111 performs the virtual buffer control using the specificdelay value input from the specific delay value determiner 113. Thethird CPB 111 assigns the specific delay value received from thespecific delay value determiner 113 to initial_cpb_removal_delay in(Formula 4). The third CPB 111 calculates tai,earliest(n) and performsthe subsequent virtual buffer control.

A third delay value calculator 112 calculates the delay value Δtg, 90(n)for each GOP based on the time information calculated by the third CPB111 according to (Formula 7). The third delay value calculator 112outputs the calculated delay value to the specific delay valuedeterminer 113.

The specific delay value determiner 113 calculates a specific delayvalue based on the delay values output from the first delay valuecalculator 109, the second delay value calculator 110 and the thirddelay value calculator 112. For example, smallest one of the delay valueobtained by the first delay value calculator 109, the delay valueobtained by the second delay value calculator 110, and the delay valueobtained by the third delay value calculator 112 is output as thespecific delay value. The specific delay value determiner 113 may outputa value smaller than the obtained delay value. The specific delay valuedeterminer 113 outputs the calculated specific delay value to themultiplexer 114. Further, the specific delay value determiner 113outputs the calculated specific delay value to the first CPB 102, thesecond CPB 105 and the third CPB 111.

The multiplexer 114 basically performs the same operation as that of themultiplexer 108 in the first embodiment. The multiplexer 114 isdifferent from the multiplexer 108 in the first embodiment in that thespecific delay value calculated by the specific delay value determiner113 is written as the delay value of the extended stream. Themultiplexer 114 performs the above operation, to be capable of makingthe delay value of the basic stream, the delay value of the extendedstream and the delay value of the three dimensional stream equal to eachother. In the H.264/AVC standard, the delay value of the extended streamis written into a suitable part of syntax defined by the H.264/AVC suchas “Buffering period SEI”. That is to say, such a part may be any partas long as the delay value of the extended stream can be read therefromin a reproducing side.

3. Operation

One example of the operation of the 3D encoding apparatus according tothe third embodiment will be described below with reference to aflowchart of FIG. 9.

(Step S901) When an input image is input, the first encoder 101 encodesthe input image to generate a basic stream. The first encoder 101outputs the generated basic stream to the multiplexer 114. Further, thefirst encoder 101 outputs a code amount and an encoding parameter of thegenerated basic stream to the first CPB 102 and the second CPB 105.

(Step S902) The first CPB 102 performs the virtual buffer control basedon the encoding parameter and the code amount of the basic streamgenerated by the first encoder 101. The first CPB 102 outputs timeinformation generated in the process of the virtual buffer control tothe first delay value calculator 109.

(Step S903) On the other hand, when an input image is input, the secondencoder 104 encodes the input image to generate an extended stream. Thesecond encoder 104 outputs the generated extended stream to themultiplexer 114. The second encoder 104 outputs a code amount and anencoding parameter of the generated extended stream to the second CPB105 and the third CPB 111.

(Step S904) The second CPB 105 performs the virtual buffer control basedon the code amount and the encoding parameter of the basic streamgenerated by the first encoder 101, and the code amount and the encodingparameter of the extended stream generated by the second encoder 104.The second CPB 105 outputs time information generated in the process ofthe virtual buffer control to the second delay value calculator 110.

(Step S905) The third CPB 111 performs the virtual buffer control basedon the code amount and the encoding parameter of the extended streamgenerated by the second encoder 104. The third CPB 111 outputs timeinformation generated in the process of the virtual buffer control tothe third delay value calculator 112.

(Step S906) The first encoder 101 and the second encoder 104 determinewhether an input image to be encoded is an input image (hereinafter,“terminal image”) positioned at the end of GOP. When the determinationis made as not being a terminal image, the sequence goes to S901, andthe first encoder 101 and the second encoder 104 encodes residual inputimages in GOP. When the determination is made as being a terminal image,the sequence goes to S907.

(Step S907) The first delay value calculator 109 calculates a delayvalue (hereinafter, “the first delay value”) of the basic stream basedon the time information output from the first CPB 102. The first delayvalue is not a specific delay value to be actually added to the basicstream. The specific delay value is calculated by the specific delayvalue determiner 113. The first delay value calculator 109 outputs thecalculated first delay value to the specific delay value determiner 113.

(Step S908) The second delay value calculator 110 calculates a delayvalue (hereinafter, “the second delay value”) to be applied to the threedimensional stream based on the time information output from the secondCPB 105. The second delay value is not a specific delay value. Thespecific delay value is calculated by the specific delay valuedeterminer 113. The second delay value calculator 110 outputs thecalculated second delay value to the specific delay value determiner113.

(Step S909) The third delay value calculator 112 calculates a delayvalue (hereinafter, “a third delay value”) to be applied to the extendedstream based on the time information output from the third CPB 111. Thethird delay value is not a specific delay value. The specific delayvalue is calculated by the specific delay value determiner 113. Thethird delay value calculator 112 outputs the calculated third delayvalue to the specific delay value determiner 113.

(Step S910) The specific delay value determiner 113 determines thesmallest delay value of the first delay value, the second delay valueand the third delay value output from the first delay value calculator109, the second delay value calculator 110 and the third delay valuecalculator 112 as a specific delay value, and outputs it to the firstCPB 102, the second CPB 105 and the third CPB 111.

(Step S911) Finally, the first CPB 102, the second CPB 105 and the thirdCPB 111 perform the virtual buffer control over GOP (n+1th GOP) next toGOP (nth GOP) of which specific delay value is obtained based on thespecific delay value output from the specific delay value determiner113.

The above operation flow is one example, and the respective steps can beshuffled. The specific delay value generated by the specific delay valuedeterminer 113 is not limited to the above value, and for example, adelay value that is smaller than the smallest delay value of the delayvalues output from the first delay value calculator 109, the seconddelay value calculator 110 and the third delay value calculator 112 maybe used as the specific delay value.

4. Conclusion

The 3D encoding apparatus according to the third embodiment includes thefirst encoder 101 and the second encoder 104 that encode input images togenerate a basic stream and an extended stream, the first CPB 102 thatperforms the buffer simulation of decoding the basic stream output fromthe first encoder 101, the second CPB 105 that performs the buffersimulation of decoding a three dimensional stream obtained from thebasic stream output from the first encoder 101 and the extended streamoutput from the second encoder 104, the third CPB 111 that performs thebuffer simulation of decoding the extended stream, the first delay valuecalculator 109 that calculates the first delay value as a delay value ofthe basic stream for each GOP based on a result of the buffer simulationperformed by the first CPB 102, the second delay value calculator 110that calculates the second delay value as a delay value of the threedimensional stream for each GOP based on a result of the buffersimulation performed by the second CPB 105, the third delay valuecalculator 112 that calculates the third delay value as a delay value ofthe extended stream for each GOP based on a result of the buffersimulation performed by the third CPB 111, and the specific delay valuedeterminer 113 that sets the specific delay value based on the firstdelay value, the second delay value and the third delay value. Thespecific delay value determiner 113 determines a value that is not morethan the smallest delay value of the first delay value, the second delayvalue and the third delay value. The first CPB 102, the second CPB 105and the third CPB 111 perform the buffer simulation of n+1th GOP basedon the specific delay value of the nth GOP in the basic stream, thethree dimensional stream and extended stream set by the specific delayvalue determiner 113.

The 3D encoding apparatus according to the third embodiment can set thespecific delay value based on the first delay value, the second delayvalue and the third delay value. The 3D encoding apparatus can feed backthe set specific delay value to the first BS, the second BS and thebuffer simulation (hereinafter, third BS) at a time of generating theextended stream. Therefore, the 3D encoding apparatus can perform thefirst BS, the second BS and the third BS in next GOP based on the samedelay values. With the simple configuration, the first delay value, thesecond delay value and the third delay value can be set to the samevalue.

Since the specific delay value is not more than the smallest one of thefirst delay value, the second delay value and the third delay value,even when the delay values of the first BS, the second BS and the thirdBS are replaced by the same delay value so that the buffer simulation isperformed, the basic stream, the three dimensional stream and theextended stream that meet the BD standard can be always generatedwithout interrupting the first BS, the second BS and the third BS.

For, example, the first CPB 102, the second CPB 105 and the third CPB111 output the information about the time of encoding input images asthe result of the buffer simulation to the first delay value calculator109, the second delay value calculator 110 and the third delay valuecalculator 112.

In such a manner, the 3D encoding apparatus can calculate the delayvalue based on the information relating to the encoding time. As aresult, since the accurate delay value can be calculated and the buffersimulation can be performed, while a deterioration in videos is beingrepressed, the basic stream, the three dimensional stream and theextended stream that meet the BD standard can be generated.

(Fourth Embodiment)

1. Outline

The second embodiment describes the configuration that the specificdelay value is calculated so that the delay value of the basic streamand the delay value of the three dimensional stream calculated based onthe cpb value are equal to each other.

In the fourth embodiment, the specific delay value is calculated so thatthe delay value of the basic stream, the delay value of the extendedstream and the delay value of the three dimensional stream calculatedbased on the cpb value are equal to each other.

2. Configuration

The 3D encoding apparatus according to a fourth embodiment will bedescribed below with reference to the drawings.

FIG. 10 is a diagram illustrating a configuration of the 3D encodingapparatus according to the fourth embodiment. Parts that have sameconfiguration as those in the second embodiment are denoted by the samereference symbols, and detailed description thereof is omitted.

The first simple CPB 301 and the second simple CPB 303 perform thevirtual buffer control using the specific cpb value output from the CPBvalue inverse converter 312 instead of the cpb value output from the CPBvalue inverse converter 306.

The first delay value calculator 307 basically performs the similaroperation to that of the first delay value calculator 302 according tothe second embodiment. The first delay value calculator 307 is differentfrom the first delay value calculator 302 according to the secondembodiment in that it outputs the calculated delay value to the specificdelay value determiner 311.

The second delay value calculator 308 basically performs the similaroperation to that of the second delay value calculator 304 according tothe second embodiment. The second delay value calculator 308 isdifferent form the second delay value calculator 304 according to thesecond embodiment in that it outputs the calculated delay value to thespecific delay value determiner 311.

A third simple CPB 309 is a virtual buffer for performing the virtualbuffer control using the cpb value based on the encoding parameter andthe code amount output from the second encoder 104. The encodingparameter represents information about the encoding including syntaxinformation. The code amount is a code amount obtained by actuallyencoding an input image through the second encoder 104 or a code amountas a predicted value. The third simple CPB 309 calculates the cpb value(hereinafter, “cpb3”) in the extended stream. The third simple CPB 309calculates cpb3(n) of an nth input image according to the followingformula.cpb3(n)=cpb3(n−1)+(BR3*T)−b3(n)  (Formula 12)

BR3 represents a maximum bit rate of the extended stream, and is, forexample, 12 Mbps. T represents an inverse number of a picture rate. Inother words, T is a value representing time from an (n−1st) input imageto an nth input image included in the extended stream. Further, b3(n)represents a code amount of the nth input image in the extended stream.That is to say, b3(n) is a code amount of the nth input image in theextended stream output from the second encoder 104. The maximum bit rateand the picture rate of the extended stream are given from thecontroller (not shown) for controlling the 3D encoding apparatusentirely. The maximum bit rate of the extended stream may be a valuethat changes according to GOPs. Further, fixed maximum bit rates may begiven to input images.

The third simple CPB 309 outputs cpb3(n) to the third delay valuecalculator 310. The third simple CPB 309 outputs cpb3(n) to the thirddelay value calculator 310 at, for example, the end of GOP.

The third simple CPB 309 uses the cpb value output from the CPB valueinverse converter 312 for the virtual buffer control over next GOP. Thatis to say, in the virtual buffer control over next GOP, a cpb valueoutput form the CPB value inverse converter 306 is used instead of thecurrent cpb value. Even if the current cpb value is replaced, the delayvalue of the basic stream, the delay value of three dimensional streamand the delay value of the extended stream can be made to be equal toeach other by the above operation of the third simple CPB 309 withoutinterrupting the virtual buffer control.

The third delay value calculator 310 calculates the delay value of theextended stream based on cpb3 output from the third simple CPB 309 andthe maximum bit rate of the extended stream given from the controller400 for controlling the 3D encoding apparatus entirely.

The third delay value calculator 310 calculates the delay value of theextended stream based on a relationship of (Formula 10). For example,3200 bits is given as cpb3, and 12 Mbps is given as the maximum bitrate. In this case, the third delay value calculator 310 calculates thedelay value to obtain 0.266 ms according to the (Formula 10).

The third delay value calculator 310 outputs the calculated delay valueto the specific delay value determiner 311.

The specific delay value determiner 311 calculates the specific delayvalue based on the delay values output from the first delay valuecalculator 307, the second delay value calculator 308 and the thirddelay value calculator 310. For example, the smallest one of the delayvalue obtained by the first delay value calculator 307, the delay valueobtained by the second delay value calculator 308 and the delay valueobtained by the third delay value calculator 310 is output to the CPBvalue inverse converter 312. The specific delay value determiner 311 mayoutput a smaller value than the smallest one of the delay value obtainedby the first delay value calculator 307, the delay value obtained by thesecond delay value calculator 308 and the delay value obtained by thethird delay value calculator 310. The specific delay value determiner311 outputs the obtained delay value to the CPB value inverse converter312. The specific delay value determiner 311 outputs the calculatedspecific delay value to the multiplexer 114.

The CPB value inverse converter 312 calculates the cpb values applied tothe first simple CPB 301, the second simple CPB 303 and the third simpleCPB 309 based on (1) the specific delay value output from the specificdelay value determiner 311, (2) the maximum bit rate of the basicstream, (3) the maximum bit rate of the three dimensional stream and (4)the maximum bit rate of the extended stream, these maximum bit ratesfrom (2) to (4) being given from the controller for controlling the 3Dencoding apparatus entirely. The CPB value inverse converter 312 outputsthe calculated cpb values to the first simple CPB 301, the second simpleCPB 303 and the third simple CPB 309.

For example, the CPB value inverse converter 312 obtains 0.125 ms as thespecific delay value from the specific delay value determiner 311. Themaximum bit rate of the basic stream is 12 Mbps, the maximum bit rate ofthe three dimensional stream is 24 Mbps, and the maximum bit rate of theextended stream is 12 Mbps. At this time, the CPB value inverseconverter 312 calculates the cpb values to be applied to the firstsimple CPB 301, the second simple CPB 303 and the third simple CPB 309according to (Formula 12).

That is to say, the CPB value inverse converter 312 calculates the cpbvalue to be applied to the first simple CPB 301 to obtain 1500 bits.Further, the CPB value inverse converter 312 calculates the cpb value tobe applied to the second simple CPB 303 to obtain 3000 bits. The CPBvalue inverse converter 312 calculates the cpb value to be applied tothe third simple CPB 309 to obtain 1500 bits.

3. Operation

One example of the operation of the 3D encoding apparatus according tothe fourth embodiment will be described below with reference to aflowchart in FIG. 11.

(Step S1101) When an input image is input, the first encoder 101 encodesthe input image to generate a basic stream. The first encoder 101outputs the generated basic stream to the multiplexer 114. Further, thefirst encoder 101 outputs a code amount and an encoding parameter of thegenerated basic stream to the first simple CPB 301 and the second simpleCPB 303.

(Step S1102) The first simple CPB 301 performs the virtual buffercontrol using the cpb value based on the encoding parameter and the codeamount of the basic stream generated by the first encoder 101.

(Step S1103) On the other hand, when an input image is input, the secondencoder 104 encodes the input image to generate an extended stream. Thesecond encoder 104 outputs the generated extended stream to themultiplexer 114. The second encoder 104 outputs a code amount and anencoding parameter of the generated extended stream to the second simpleCPB 303 and the third simple CPB 309.

(Step S1104) The second simple CPB 303 performs the virtual buffercontrol using the cpb value based on the code amount and the encodingparameter of the basic stream generated by the first encoder 101, andthe code amount and the encoding parameter of the extended streamgenerated by the second encoder 104.

(Step S1105) The third simple CPB 309 performs the virtual buffercontrol using the cpb value based on the code amount and the encodingparameter of the extended stream generated by the second encoder 104.

(Step S1106) The first encoder 101 and the second encoder 104 determinewhether an input image to be encoded is an input image (hereinafter,“terminal image”) positioned at the end of GOP. When the determinationis made as not being the terminal image, the sequence goes to S1101, andthe first encoder 101 and the second encoder 104 encodes residual inputimages in GOP. When the determination is made as being the terminalimage, the first simple CPB 301 outputs the cpb value at the time of theend of GOP to the first delay value calculator 307. Further, the secondsimple CPB 303 outputs the cpb value at the time of the end of GOP tothe second delay value calculator 308. The third simple CPB 309 outputsthe cpb value at the time of the end of GOP to the third delay valuecalculator 310. The process goes to the second delay value calculator1107.

(Step S1107) The first delay value calculator 307 calculates a delayvalue (hereinafter, “the first delay value”) of the basic stream basedon the cpb value output from the first simple CPB 301. The first delayvalue is not a specific delay value to be actually added to the basicstream. The specific delay value to be actually added is calculated bythe specific delay value determiner 311. The first delay valuecalculator 307 outputs the calculated first delay value to the specificdelay value determiner 311.

(Step S1108) The second delay value calculator 308 calculates a delayvalue (hereinafter, “a second delay value”) to be applied to the threedimensional stream based on the cpb value output from the second simpleCPB 303. The second delay value is not the specific delay value. Thespecific delay value to be actually added is calculated by the specificdelay value determiner 311. The second delay value calculator 308outputs the calculated second delay value to the specific delay valuedeterminer 311.

(Step S1109) The third delay value calculator 310 calculates a delayvalue (hereinafter, “a third delay value”) to be applied to the extendedstream based on the cpb value output from the third simple CPB 309. Thethird delay value is not a specific delay value. The specific delayvalue to be actually added is calculated by the specific delay valuedeterminer 311. The third delay value calculator 310 outputs thecalculated third delay value to the specific delay value determiner 311.

(Step S1110) The specific delay value determiner 311 determines thesmallest one of the first delay value, the second delay value and thethird delay value output from the first delay value calculator 307, thesecond delay value calculator 308 and the third delay value calculator310 as the specific delay value, and outputs it to the CPB value inverseconverter 312.

(Step S1111) The CPB value inverse converter 312 calculates the cpbvalues to be used in the first simple CPB 301, the second simple CPB 303and the third simple CPB 309 based on the specific delay value outputfrom the specific delay value determiner 311. The CPB value inverseconverter 312 outputs the calculated cpb values to the first simple CPB301, the second simple CPB 303 and the third simple CPB 309respectively.

(Step S1112) Finally, the first simple CPB 301, the second simple CPB303 and the third simple CPB 309 perform the virtual buffer control overGOP (n+1th GOP) next to GOP (nth GOP) of which specific delay value isobtained based on the cpb value output from the CPB value inverseconverter 312.

The above operation flow is one example, and the respective steps can beshuffled. The specific delay value to be generated at S1110 is notlimited to the above value, and for example, a smaller delay value thanthe first delay value, the second delay value and the third delay valueoutput from the first delay value calculator 307, the second delay valuecalculator 308 and the third delay value calculator 310 may be used asthe specific delay value.

4. Conclusion

The 3D encoding apparatus according to the embodiment includes the firstencoder 101 and the second encoder 104 that encode input images togenerate and output a basic stream and an extended stream, the firstsimple CPB 301 that performs the buffer simulation of decoding the basicstream, the second simple CPB 303 that performs the buffer simulation ofdecoding a three dimensional stream obtained based on the basic streamoutput from the first encoder 101 and the extended stream output fromthe second encoder 104, the third simple CPB 309 that performs thebuffer simulation of decoding the extended stream, the first delay valuecalculator 307 that calculates a first delay value as a delay value ofthe basic stream for each GOP based on a result of the buffer simulationperformed by the first simple CPB 301, the second delay value calculator308 that calculates a second delay value as a delay value of the threedimensional stream for each GOP based on a result of the buffersimulation performed by the second simple CPB 303, the third delay valuecalculator 310 that calculates the third delay value as a delay value ofthe extended stream for each GOP based on a result of the buffersimulation performed by the third simple CPB 309, and the specific delayvalue determiner 311 that sets a specific delay value for each GOP basedon the first delay value, the second delay value and the third delayvalue. The specific delay value determiner 113 determines a value notmore than the smallest one of the first delay value, the second delayvalue and the third delay value. The first simple CPB 301, the secondsimple CPB 303 and the third simple CPB 309 perform the buffersimulation for n+1th GOP based on the specific delay values of nth GOPin the basic stream, the three dimensional stream and the extendedstream set by the specific delay value determiner 311. The first simpleCPB 301 outputs a buffer accumulation amount in the first simple CPB 301as a result of the buffer simulation to the first delay value calculator307. The second simple CPB 303 outputs a buffer accumulation amount inthe second simple CPB 303 as a result of the buffer simulation to thesecond delay value calculator 308. The third simple CPB 309 outputs abuffer accumulation amount in the third simple CPB 309 as a result ofthe buffer simulation to the third delay value calculator 310.

In such a manner, even when the 3D encoding apparatus according to thefourth embodiment performs the buffer simulation using the bufferaccumulation amount, the delay value to be added to the basic stream,the delay value to be added to the three dimensional stream and thedelay value to be added to the extended stream can be made to be equalto each other with the simple configuration. As a result, even when thebuffer simulation is performed in the simple configuration, the basicstream, the three dimensional stream and the extended stream that meetthe BD standard can be generated.

Since the specific delay value is not more than the smallest one of thefirst delay value, the second delay value and the third delay value,even when the delay values of the first BS, the second BS and the thirdBS are replaced by the same delay value and the buffer simulation isperformed, the basic stream, the three dimensional stream and theextended stream that meet the BD standard can be always generatedwithout interrupting the first BS, the second BS and the third BS.

For, example, when the specific delay value output from a specific delayvalue determiner 311 is input, the first simple CPB 301 modifies thebuffer accumulation amount of the first simple CPB 301 at the input timebased on the specific delay value, and performs the buffer simulationfor GOP (n+1th GOP) next to GOP (nth GOP) of which specific delay valueis obtained. When a specific delay value output from the specific delayvalue determiner 311 is input, the second simple CPB 303 modifies thebuffer accumulation amount of the second simple CPB 303 at the inputtime based on the specific delay value, and performs the buffersimulation for GOP (n+1th GOP) next to GOP (nth GOP) of which specificdelay value is obtained. When a specific delay value output from thespecific delay value determiner 311 is input, the third simple CPB 309modifies the buffer accumulation amount of the third simple CPB 309 atthe input time based on the specific delay value, and performs thebuffer simulation for GOP (n+1th GOP) next to GOP (nth GOP) of whichspecific delay value is obtained.

In such a manner, the buffer accumulation amounts in the first simpleCPB 301, the second simple CPB 303 and the third simple CPB 309 can bemodified according to the setting of the specific delay value in the 3Dencoding apparatus. As a result, even when the first delay value, thesecond delay value and the third delay value are changed into thespecific delay value, the 3D encoding apparatus can generate the basicstream and the three dimensional stream that meet the BD standardwithout interrupting the buffer simulation using the buffer accumulationamount.

Industrial Applicability

The 3D encoding apparatus of the present disclosure can be applied todigital television, video players, personal computers and mobiletelephones for encoding video signals.

What is claimed is:
 1. A 3D encoding apparatus, comprising: an encoderthat encodes an input image and generates and outputs a threedimensional stream including a basic stream and an extended streamincluding a plurality of image groups; a first virtual buffer thatperforms a buffer simulation of decoding the basic stream output fromthe encoder; a second virtual buffer that performs a buffer simulationof decoding the three dimensional stream output from the encoder; afirst calculator that calculates a first delay value as a delay value ofthe basic stream for each image group based on a result of the buffersimulation performed by the first virtual buffer, a second calculatorthat calculates a second delay value as a delay value of the threedimensional stream for each image group based on a result of the buffersimulation performed by the second virtual buffer; and a setter thatsets a specific delay value for each image group based on the calculatedfirst delay value and second delay value, wherein the setter sets thespecific delay value to a value not more than smaller one of the firstdelay value and the second delay value, the first virtual buffer and thesecond virtual buffer perform the buffer simulation for n+1th imagegroup based on information about the specific delay value of the nthimage group set by the setter in the basic stream and three dimensionalstream.
 2. The 3D encoding apparatus according to claim 1, wherein thefirst virtual buffer and the second virtual buffer output informationabout time at which the input image is encoded as the result of thebuffer simulation to the first calculator and the second calculator. 3.The 3D encoding apparatus according to claim 1, wherein the firstvirtual buffer outputs an accumulation amount of data in the firstvirtual buffer as the result of the buffer simulation to the firstcalculator, and the second virtual buffer outputs an accumulation amountof data in the second virtual buffer as the result of the buffersimulation to the second calculator.
 4. The 3D encoding apparatusaccording to claim 3, wherein when the specific delay value output fromthe setter is input, the first virtual buffer modifies the accumulationamount of the first virtual buffer at the input time of the specificdelay value based on information about the specific delay value andperforms the buffer simulation for the n+1th image group, when thespecific delay value output from the setter is input, the second virtualbuffer modifies the accumulation amount of the second virtual buffer atthe input time of the specific delay value based on information aboutthe specific delay value, and performs the buffer simulation for then+1th image group.
 5. The 3D encoding apparatus according to claim 1,further comprising: a third virtual buffer that performs a buffersimulation of decoding the extended stream output from the encoder; anda third calculator that calculates a third delay value as a delay valueof the extended stream for each image group based on a result of thebuffer simulation performed by the third virtual buffer, wherein thesetter sets the specific delay value for each image group based on thefirst delay value, the second delay value and the third delay value, thesetter sets a value not more than the smallest one of the first delayvalue, the second delay value and the third delay value as the specificdelay value, and the first virtual buffer, the second virtual buffer andthe third virtual buffer perform the buffer simulation for the n+1 imagegroup in the basic stream, the three dimensional stream and the extendedstream based on the information about the specific delay value set bythe setter.
 6. The 3D encoding apparatus according to claim 5, whereinthe first virtual buffer, the second virtual buffer and the thirdvirtual buffer output information about the time at which the inputimage is encoded as results of the buffer simulation to the firstcalculator, the second calculator and the third calculator,respectively.
 7. The 3D encoding apparatus according to claim 5, whereinthe first virtual buffer outputs an accumulation amount of data in thefirst virtual buffer as a result of the buffer simulation to the firstcalculator, the second virtual buffer outputs an accumulation amount ofdata in the second virtual buffer as a result of the buffer simulationto the second calculator, and the third virtual buffer outputs anaccumulation amount of data in the third virtual buffer as a result ofthe buffer simulation to the third calculator.
 8. The 3D encodingapparatus according to claim 7, wherein when the specific delay valueoutput from the setter is input, the first virtual buffer modifies theaccumulation amount of the first virtual buffer at the input time of thespecific delay value based on information about the specific delayvalue, and performs the buffer simulation for the n+1th image group,when the specific delay value output from the setter is input, thesecond virtual buffer modifies the accumulation amount of the secondvirtual buffer at the input time of the specific delay value based oninformation about the specific delay value, and performs the buffersimulation for the n+1th image group, and when the specific delay valueoutput from the setter is input, the third virtual buffer modifies theaccumulation amount of the third virtual buffer at the input time of thespecific delay value based on information about the specific delayvalue, and performs the buffer simulation for n+1th image group.
 9. The3D encoding apparatus according to claim 1, wherein the basic stream isa stream obtained by encoding one of a left-eye image and a right-eyeimage in a 3D video, and the extended stream is a stream obtained byencoding the other image in the 3D video in view of a difference betweenthe one image and the other image.